WORLD INTELLECTUAL PROPERTY ORGANIZATION 

International Bureau 




PCX 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification ^ : 

GllB 27/031, 27/34, H04N 5/262, G06F 
3/033, 9/44, 17/30 



Al 



(11) International Publication Number: WO 98/06098 

(43) International Publication Date: 12 February 1998 (12,02.98) 



(21) International Application Number: 

(22) International Filing Date: 



PCT/US97/I3055 



6 August 1997 (06,08.97) 



(30) Priority Data: 

60/023.359 



6 August 1996 (06.08.96) 



US 



(71) Applicant: APPLIED MAGIC, INC. fUS/US]; 6078 Corte Del 

Cedro. Carlsbad, CA 92009-1514 (US). 

(72) Inventors: NEWMAN, David, Anthony; 2421 Union Street, 

San Diego, CA 92101 (US). WALLIN, Robert, Lee; 815 
Caminito Verde, Carlsbad, CA 92009 (US). 

(74) Agent: ALTMAN. Daniel, E.; Knobbe, Martens, Olson & Bear, 
16th floor, 620 Newport Center Drive, Newport Beach, CA 
92660 (US). 



(81) Designated States: AL, AM, AT, AU, AZ, BA, BB, BG, BR, 
BY, CA. CH. CN, CU, CZ, DE, DK, EE, ES, Fl, GB, GE, 
HU, IL, IS, JP, KE, KG, KP. KR, KZ, LC, LK, LR, LS, LT, 
LU» LV, MD. MG, MK, MN, MW, MX, NO, NZ, PL, PT, 
RO, RU, SD, SE, SG, SI, SK, TJ, TM, TR, TT, UA, UG, 
UZ, VN, ARIPO patent (GH, KE, LS, MW, SD, SZ. UG, 
ZW), Eurasian patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, 
TM), European patent (AT, BE, CH, DE, DK, ES, FI, FR, 
GB, GR, IE, IT, LU, MC, NL, PT, SE), OAPI patent (BP. 
BJ. CF, CG, CI, CM, GA, GN, ML, MR. NE. SN. TD. TG). 



Published 

With international search report. 

Before the expiration of the time limit for amending the 
claims and to be republished in the event of the receipt of 
amendments. 



(54) Title: NON-LINEAR EDITING SYSTEM FOR HOME ENTIiRTAINMENT ENVIRONMENTS 




NGN-LINEAR EDITOR 

(57) Abstract 

A non-linear editing system for home audio and video applications includes a compression/decompression engine, a high capacity 
storage device and a media editor that provides point and click audio and video editing functionality, including rccoreling, playback 
and special effects, such as real lime gamma correction, color effects, 2D effects and real time fades, using a time-line system. The 
compression/decompression engine includes electronic circuitry designed to implement high speed data compression and decompression 
using JPEG, MPEG, or wavelet techniques. The high capacity storage device typically comprises internal and external non-linear magnetic 
storage devices, such as Enhanced IDE or SCSI hard drives, although other non-linear storage devices, such as magneto-optical or optical 
disk drives may be used. Similarly, the system includes input/output (I/O) capability for video in composite NTSC or PAL fonnats including 
SVHS resolutiorxs, analog and digital stereo audio in 16-bit CD format, multimedia inputs, such as clip art and synchronized audio/video, 
from CD-ROMs and DVDs, a Musical Instrument Digital Interface (MIDI) and Internet connectivity through strandard telephone lines using 
a modem and through cable TV lines. The system does not require the use of a computing device, such as a personal computer, to perform 
its non-linear editing functions. 



FOR THE PURPOSES OF INFORMATION ONLY 



Codes used to identify States party to the PCT on tlie front pages of pamphlets publishing international applications under the PCT. 



AL 


Albania 


ES 


Spain 


LS 


Lesotho 


SI 


Slovenia 


AM 


Armenia 


Fl 


Finland 


LT 


Lithuania 


SK 


Slovakia 


AT 


Austria 


FR 


Frsmcc 


LU 


Luxembourg 


SN 


Senegal 


AU 


Australia 


GA 


Gabon 


LV 


Latvia 


sz 


Swaziland 


AZ 


Azerbaijan 


GB 


United Kingdom 


MC 


Monaco 


TD 


Chad 


BA 


Bosnia and Herzegovina 


GE 


Georgia 


MD 


Republic of Moldova 


TG 


Togo 


BB 


Barbados 


Gil 


Ghana 


MG 


Madagascar 


TJ 


Tajikistan 


BE 


Belgium 


GN 


Guinea 


MK 


The former Yugoslav 


TM 


Turkmenistan 


BF 


Burkina Faso 


GR 


Greece 




Republic of Macedonia 


TR 


Turkey 


BG 


Bulgaria 


HU 


Hungary 


ML 


Mali 


TT 


Trinidad and Tobago 


BJ 


Benin 


IE 


Ireland 


MN 


Mongolia 


UA 


Ukraine 


BR 


Brazil 


IL 


Israel 


MR 


Mauritania 


UG 


Uganda 


BY 


Belanui 


IS 


Iceland 


MW 


Malawi 


US 


United States of America 


CA 


Canada 


IT 


Italy 


MX 


Mexico 


uz 


Uzbekistan 


CF 


Central African Republic 


JP 


Japan 


NE 


Niger 


VN 


Viet Nam 


CG 


Congo 


KE 


Kenya 


NL 


Netherlands 


YU 


Yugoslavia 


CII 


Switzerland 


KG 


Kyrgyzsian 


NO 


Norway 


ZW 


Zimbabwe 


CI 


COie d'TvcMrc 


KP 


Democratic People's 


NZ 


New Zealand 






CM 


Cameroon 




Republic of Korea 


PL 


Poland 






CN 


China 


KR 


Republic of Korea 


PT 


Portugal 






cu 


Cuba 


KZ 


Kazakstan 


RO 


Romania 






cz 


Czech Republic 


LC 


Saint Lucia 


RU 


Russian Federation 






DE 


Germany 


LI 


Liechtenstein 


SD 


Sudan 






DK 


Denmark 


LK 


Sri Lanka 


SE 


Sweden 






EE 


Estonia 


LR 


Liberia 


SG 


Singapore 







wo 98/06098 PCT/US97/13055 

-1- 

Non-Linear Editing System for Home Entertainment Environments 



Background of the invention 

Field of the Invention 

The invention relates to non-linear editing systems and, more particularly, to consumer systems for the 
storage, editing and retrieval of audio/visual information. 
Description of the Related Technology 

Linear video editing systems, such as those for videotape and photographic film, are old in the art. In 
contrast, current personal computer (PC) systems, such as those of the Apple® Macintosh® and Intel® architecture, 
offer non-linear video editing systems. Non linear editing on computer oriented systems involves digitizing analog 
media data recorded from a linear source, such as videotape or film, and storing the digitized media data on a 
storage device, such as a magnetic disk drive. Once digitized, the non-linear editing system permits rapid access to 
media data at any point in the linear sequence for subsequent manipulation of media data portions into any order. 
For example, non linear editing systems enable the combination of audio clips with other video clips and the formation 
of new clips using portions of other clips. 

The non linear video editing capability typically resides in a plug-in card for the NuBus or PCI expansion bus 
of a Macintosh architecture PC or the ISA, EISA or PCI expansion bus of an Intel architecture PC. These non-linear 
video editing systems typically use compression techniques developed by the Joint Photographic Experts Group (JPEG) 
or the Motion Picture Experts Group (MPEG). For example, in U.S. Patent No. 5,577,190, Peters discloses a media 
editing system that receives, digitizes, stores and edits audio and video source material, using JPEG compression, 
for later manipulation by a computer, such as an Apple Macintosh. Similarly, in U.S. Patent No. 5,508,940, 
Rossmere, et al., disclose a multimedia random access audio/video editing system including a main control unit that 
receives data and commands over a SCSI bus from a personal computer having an analog I/O board coupled to audio 
and video processing boards using JPEG compression. Moreover, Reber et al. disclose a system and method for the 
management of media data in a non-linear editing system that enables dynamic linking of digitized media data with 
a specific source reference at run time in U.S. Patent No. 5,584,006. Lastly, in U.S. Patent No. 5,438,423, Lynch, 
et aL, disclose a system and method for continuously storing the video content of a program, using JPEG or MPEG 
compression, in a recirculating random access buffer having sufficient capacity to store a substantial portion of the 
program. 

Unfortunately, consumers currently have no cost effective alternatives for enhancement of their camcorder 
movies and digital pictures without having to incur substantial costs to purchase a personal computer with a high 
resolution computer graphics monitor, associated add in cards and software for non-linear editing. In addition, 
conventional non-linear editing systems are designed for expert users, such as a professional movie editor, to edit 
a large number of unrelated movie clips stored on the same linear film or videotape. Thus, conventional non-linear 
editing system tools are complex and require a high degree of manual interaction and configuration to produce a final 
edited result. In contrast, consumers often capture closely related events, such as vacations and birthday parties. 
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on videotape using their camcorders. To edit these camcorder tapes, consumers require easy to use non-linear editing 
systems that facilitate editing without a high degree of computer or editing skill and time spent to configure plug-in 
cards and software. Similarly, manufacturers currently have no viable, cost effective means for incorporating non- 
linear editing functionality into their home entertainment components because currently available non-linear editing 
systems are specifically adapted for personal computer plug-in applications and functionality, instead of home 
entertainment components. Conventional non linear editing systems, such as Adobe® Premiere® provide user 
interfaces designed for rendering on high resolution non interlaced computer graphics monitors. Although computer 
graphics monitors are viewed from a distance of one to two feet, consumers must often zoom-in on portions of the 
user interface to perform non-linear editing functions. In contrast, conventional television sets having low resolution 
interlaced displays, commonly available in home entertainment environments, render poor quality images of these user 
interfaces designed for high resolution graphics monitors. To compound matters, consumers often view their television 
sets from a distance of several feet, where a poor quality rendition of a user interface severely impedes its use. 
Consumers require a non linear editing system adapted for use with conventional television sets. 

Lastly, current personal computer based non-linear editing systems are specifically adapted to an RGB (red, 
green and blue) color space used for non-interlaced computer graphics monitors or CRTs. However, RGB is a poor 
choice for representing real-world images because equal bandwidths are needed to describe each color component 
while the human eye is more sensitive to luminance (the Y or black and white component) than color components 
(the U and V components). Similarly, equal bandwidth provides the same pixel depth and display resolution for each 
color component, which dramatically increases the cost of non-linear editing systems due to the need for expensive 
dual port video RAMs, instead of low cost synchronous DRAMs, to support the high bandwidth requirements. 
Moreover, conventional non linear editing systems increase system cost by using additional dedicated processors, such 
as audio DSPs or the CPU of the personal computer, for audio editing functions. In summary, consumers have a 
substantial unmet need for cost effective, non-linear editing systems for home audio and video applications that do 
not require the use or purchase of an expensive personal computer. 

Summarv of the Invention 

The present invention provides an economical solution for the incorporation and editing of hypermedia, such 
as motion pictures, music, animation and photographs, into a wide variety of present and future platforms. By 
eliminating the need for a costly personal computer, the present invention enables the Incorporation of conventional 
home entertainment components, such as VCRs, camcorders and compact disc players, into an economical, stand- 
alone, non-linear hypermedia editing system. The present invention allows consumers to capture hypermedia from 
real time on line sources, such as broadcast radio and television, pay per view cable/satellite television services and 
the World Wide Web portion of the Internet, as well as off-line sources, such as video cassette tapes, laserdiscs, 
DVDs and compact discs. Analog hypermedia is digitized and may be compressed for storage. Consumers may replay 
the captured hypermedia In addition to selectively capturing and manipulating hypermedia portions, or clips, using the 
graphical user kiterf ace (GUI) of the present invention. Captured clips appear as icons on the GUI and consumers 
may combine captured clips by manipulating their respective icons to effect a wide variety of editing functions, such 
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as fades, dissolves, wipes, and animated effects. Moreover, consumers may also use the point, click, drag and drop 
functionality of the GUI to integrate captured clips onto a timeline to form a motion picture clip. In a similar manner, 
consumers may edit and incorporate still photographs, audio, text and other data for incorporation into a clip. Lastly, 
the present invention provides for a wide variety of output mediums for the edited results, such as television sets, 
color printers, videotapes, DVDs, computer displays and audio speakers. 

One aspect of the present invention includes a non-linear editing system comprising a remote control to 
provide command signals, a main unit, responsive to the command signals, having a non linear editor to receive, 
access, and edit hypermedia and a direct access storage device to capture the hypermedia, and an output device in 
communication with the main unit to receive and play the hypermedia. 

Another aspect of the present invention includes a non-linear editor comprising a bus; a processor, in 
communication with the bus, to control access to the bus; a memory in communication with the processor; a 
compression engine, in communication with the bus, to compress and decompress hypermedia; a storage, in 
communication with the bus, to capture and provide direct access to the compressed hypermedia; and a media editor, 
in communication with the compression engine and the bus, the media editor and the compression engine receiving 
the hypermedia, wherein the media editor manipulates the hypermedia. 

Yet another aspect of the present invention includes a method of operating a non-linear editing system 
comprising the steps of capturing hypermedia, automatically loading the previously captured hypermedia and providing 
a storyboard to manipulate the previously captured hypermedia. 

Lastly, yet another aspect of the present invention includes a method of editing hypermedia in a non linear 
editing system comprising the steps of capturing a plurality of hypermedia portions to a storyboard and automatically 
providing a transition between a pair of hypermedia portions within the plurality of hypermedia portions on the 
storyboard. 

Brief Description of the Drawings 
Figure 1 is a block diagram illustrating an example of an environment for practicing the present invention. 
Figure 2a is a front side eievational view of an embodiment of a remote control of the present Invention. 
Figure 2b is a top plan view of the remote control shown in Figure 2a. 

Figure 3 is a block diagram illustrating the major functional units of the remote control shown in Figure 2. 
Figure 4a is a front side eievational view of an embodiment of a main unit of the present invention. 
Figure 4b is a cutaway front side eievational view of an embodiment of an open connector panel of the 
main unit of Figure 4a. 

Figure 4c is a rear side eievational view of the main unit shown in Figure 4a. 

Figure 5 is a block diagram illustrating the major functional units of a main unit of the present invention. 
Figure 6 is a block diagram illustrating the architecture of a media editor of the present invention. 
Figure 7 is a flowchart illustrating a method of operating the non-linear editing system of the present 

invention. 

Figure 8 is a flowchart illustrating a method of manipulating hypermedia of the present invention. 
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Figure 9 illustrates an embodiment of a graphical user interface for the capture of hypermedia. 
Figure 10 illustrates an embodiment of a graphical user interface having a storyboard. 
Figure 1 1 illustrates an embodiment of a graphical user interface for the creation of a transition. 
Figure 12 illustrates an embodiment of a graphical user interface for the creation of graphics. 
5 Figure 13 is a task flow diagram illustrating the tasks a consumer can perform using the graphical user 

interface of the present invention. 

Figure 14 is a diagram illustrating the display flow of the graphical user interface of Figure 9. 

Detailed Description of the Preferred Embodiments 
The following detailed description of the presently preferred embodiments presents a description of certain 
10 specific embodiments to assist in understanding the claims. However, one may practice the present invention in a 
multitude of different embodiments as defined and covered by the claims. 

For convenience, the description comprises four sections: the Non-Linear Editing System, the Media Editor, 
Operation of the Non linear Editing System and Summary. The first section describes the non linear editing system 
of the present invention, the next section provides an overview of the media editor of the present invention, the 
15 following section describes the operation of the non linear editing system and the remaining section summarizes 
advantageous features of the present invention. 

The term hypermedia refers to the integration of text, graphics, sound, video, and other data, or any 
combination into a primarily associative system for information presentation, storage and retrieval. For example, 
hypermedia includes motion pictures, music, animation and photographs. Hypermedia environments enable users to 
20 make associations between topics. For example, a hypermedia presentation on navigation may include associations, 
or links, to such topics as astronomy, bird migration, geography, satellites and radar. Multimedia, the combination 
of sound, graphics, animation and video information, is related to hypermedia in that hypermedia combines the 
elements of multimedia with associations to link the information. 
I. Non-Linear Editing System 
25 Figure 1 illustrates an environment for practicing the present invention. A non-linear editing system 100 

of the present invention communicates with a network 102. Network devices may include computing devices, cable 
and satellite TV tuners, Web television sets, wireless phones and information kiosks, among others. Computing 
devices communicating with the network 102 may include clients, such as a network computer and a mobile 
computer 110, and servers 112, such as a wide area network 114 including a plurality of local area networks 116, 
30 each having associated computing devices. To obtain hypermedia, such as television programming and Internet data, 
network devices communicate with the network 102 using a network connection 104. Network connections 104 
may include wired links, such as those connected to the public switched telephone network (PSTN) and a cable 
television provider, and wireless links 106, such as cellular phones, mobile computer cellular digital packet data 
(CDPD) modems and satellite dishes 108. 
35 The non linear editing system 100 includes an output device 120, such as a home television set, a main 

unit 122, in communication with the output device 120, provkling the non-linear editing functions and a remote 
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control 124. The non-linear editing system may also include hypermedia recording devices 126, such as a video 
cassette recorder (VCR), video camera, camcorder or digital video disk (DVD) recorder. The remote control 124 
communicates all editing and playback functions to the main unit 122 for display on the output device 120 preferably 
using wireless techniques, such as infrared or radio waves. However, the remote control 124 may also communicate 
with the main unit 122 using a wired connection. 

Figure 2a illustrates a front side view of one presently preferred embodiment of the remote control 124. 
Referring now to the top view of Figure 2b, the remote control 124 includes a keyboard 130 having a plurality of 
keys or buttons to communicate editing functions to the main unit 122 (Figure 1), The keyboard 130 provides for 
generation of alphanumeric characters, such as those used in home video titles, flying text, logos or graphics. The 
remote control 124 may also provide special keys 132, such as cut, paste and help buttons, and a pointing device 
134, such as a mouse, touch pad, track ball or light pen. The pointing device 134 provides for control of the on- 
screen cursor on the viewing device 120 (Figure 1) to select, scroll or move along a selected path. Moreover, the 
remote control 124 may also include a jog/shuttle wheel 136 enabling rapid selection of an individual frame of a 
motion picture. In a preferred embodiment, the remote control 124 includes a configurable device capable of 
operating as a pointing device 134 and a jog/shuttle wheel 136. Using software, a user may configure the pressure 
sensitive buttons 134, 136 of the configurable devices to operate either as a pointing device or as a jog/shuttle 
wheel. When operating as a pointing device, the on screen cursor acceleration is proportional to the pressure of user 
exertion on the button 134, 136. Advantageously, a configurable device enables proficient operation of the pointing 
device and the jog/shuttle wheel by both left handed and right handed users. Lastly, the remote control 124 may 
accept power from a portable power source, such as rechargeable and non-rechargeable batteries, as well as AC 
power from a wall socket. 

Referring now to Figure 3, a block diagram illustrates the major functional units of the remote control 124. 
The remote control 124 includes a keyboard processor 142, such as the UR5HCSPI keyboard controller available from 
USAR Systems, which receives power from a power system 146, such as a battery or AC power source, and 
processes electrical signals representing user commands from a keyboard 144. The keyboard processor 142 may 
also accept electrical signals representing user commands from a pointing device 148 and a jog/shuttle wheel 150. 
Lastly, the keyboard processor 142 may provide its output signal to a wireless transmitter 152, which provides a 
wireless signal for communication with the main unit 122 (Figure 1). 

Figure 4a illustrates a front view of one presently preferred embodiment of the main unit 122 of the present 
invention. Figure 4b provides a cutaway view of an embodiment of an open connector panel 160 of the main unit 
of Figure 4a. The connector panel 160 includes a video signal input 162, such as an RCA receptacle and a S video 
receptacle, an audio signal input 164, such as left and right channel RCA receptacles, a microphone input 166 and 
a headphone input 168 having a volume adjustment. Lastly, Figure 4c illustrates a rear view of an embodiment of 
the main unit 122. in a preferred embodiment, the main unit 122 includes a SCSI port 172, an RS232 serial port 
174 and a parallel port 176. The main unit 122 may also include a Universal Serial Bus (USB) connector, a PC Card 
connector and an IrDA compliant infrared port for communication with with various network devices. Moreover, the 
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main unit 122 preferably includes input and output connectors for coaxial cable 178, a composite video input and 
output connector 180 for video signals from a VCR or video recorder, S video input and output connectors 182, 
left/right channel audio output connectors 184 for a television set, left/right channel audio output connectors 186 
for a VCR, and left/right channel audio input connectors 188 for an auxiliary audio component, such as a compact 
5 disc player. 

Referring now to Figure 5, a block diagram illustrates the architecture of the non-linear editor 200. The 
non linear editor 200 receives hypermedia input 202 from a variety of on-line and off-line sources in a wide variety 
of formats. For example, the non-linear editor may receive separate analog YUV video signals and stereo audio 
signals from a VCR or multiplexed MPEG data packets from a digital satellite broadcaster. Thus, the non-linear editor 

10 200 may include an optional decoder 204 having an analog to digital converter to convert the hypermedia input 202 
into compatible digital data formats, such as CCIR 601 and CCIR 656 compliant bit streams or serial digital audio 
bit streams, for editing and storage. For example, the decoder 204 may convert encoded scrambled analog signal 
into decoded descrambled compatible digital data formats having separate audio and video components for subsequent 
use. Similarly, the non linear editor 200 may include an optional encoder 206 having a digital to analog converter 

15 to convert compatible digital data format into a hypermedia output 208, For example, the encoder 206 may convert 
a compatible digital data format, such as a 24 bit RGB color bitmap file, into a hypermedia output 208 including an 
analog YUV signal for an interlaced television display with a separate synchronized analog stereo signal for home 
speakers or headphones. 

The non linear editor 200 includes a media editor 210 in communication with a compression engine 212, 
20 which compresses and decompresses data, a bus 214, and an optional media buffer 216 for temporary storage. The 
non-linear editor 200 likewise includes a processor 218 in communication with the bus 214, a memory 220 and a 
storage 222. The media editor 210 and the compression engine 212 simultaneously receive the hypermedia input 
202. In this manner, the compression engine 212 compresses and transfers the hypermedia input 202 through the 
bus 214 for storing in the storage 222 white the media editor 210, which performs non-linear editing functions, 
25 advantageously provides for the output and manipulation of the hypermedia input 202. The processor 218 
administers access to the bus 214 and to the storage 222 and uses the memory 220 as a local storage area for 
program instructions and as a buffer for data. Thus, the media editor 210 communicates with the processor 218 
and the storage 222 via the bus 214. The processor 218 also supports communication interfaces (not shown), such 
as SCSI ports, serial ports, parallel ports, IrDA compliant infrared ports, USB ports. Musical Instrument Digital 
30 Interfaces (MIDI), PCMCIA/PC Card interfaces, IEEE 1394 Firewire and Smart Card interfaces, for additional network 
and hypermedia devices. 

In one presently preferred embodiment, the compression engine 212 provides data compression and 
decompression using wavelet technkiues. In another preferred embodiment, the compression engine 21 2 provides data 
compression and decompression using discrete cosine transform (OCT) techniques, such as JPEG, MPEG-1, MPEG-2 
35 and MPEG4. Similarly, the processor 218 preferably comprises a conventional embedded controller, such as a 
Hitachi SH7032, IBM PowerPC 403 or an Intel i860, although a general purpose microprocessor may be used. In 
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addition, the memory 220 may preferably be embodied as one or more of various types of electronic memory, such 
as DRAM, SRAM, ROM, EPROM, EEPROM and flash memory. Similarly, the storage 222 may preferably be embodied 
as one or more of various types of direct access non-volatile storage devices, such as magnetic, magneto-optic and 
optical disk drives. As is well known in the technology, the bus 214 comprises a set of conductors, or wires, for 
the transfer of data and addresses among the compression engine 212, the media editor 210, the processor 218 
and the storage 222. In one presently preferred embodiment, the bus 214 operates at 33MHz and transfers 32 bits 
of information in each cycle. 

The dual paths of the hypermedia input 202 enable simultaneous viewing and editing of hypermedia during 
capture. For example, a hypermedia input 202 comprising an MPEG compressed digital video bit stream is captured 
directly to the storage 222 by transfer through the bus 214 without modification by the compression engine 212 
since the hypermedia input 202, an MPEG bit stream, is already compressed. In parallel, the media editor 210 
concurrently receives the hypermedia input 202, determines the hypermedia input 202 is a compressed MPEG bit 
stream, and communicates with the compression engine 212 to decompress the MPEG bit stream for editing and 
output to an output device, such as a television set. In contrast to conventional personal computer expansion bus 
devices, the dual paths of the present invention eliminate the need to transfer large amounts of data from the 
storage 222 through the bus 214 to the media editor 210 during hypermedia capture. As a consequence, the dual 
path design likewise reduces the need for additional memory, bandwidth and processor computing power, thereby 
reducing system cost substantially. 
II. The Media Editor 

Referring now to Figure 6, a block diagram illustrates the architecture of the media editor 210. As 
discussed above, hypermedia input 202 (Figure 5) is simultaneously compressed to the storage 222 while streaming 
to the media editor 210 via a video controller 252. To prevent displayed images from jumping when viewed, the 
video controller 252 synchronizes the video output signals independently of the input video signals. Thus, the input 
horizontal and input vertical synchronization signals fed across lines 254, 256 are not used to control the output 
horizontal and output vertical synchronization signals fed across lines 258, 260. However, independent frame 
synchronization often results in a "tearing" distortion of unmanipulated, or pass-through, video images. The present 
invention eliminates "tearing" distortions through a design whereby hypermedia input 202 and hypermedia output 208 
share the same master pixel clock 250. Thus, the media editor 210 produces a stable video output from the video 
portion of a hypermedia input 202. which may include unstable and distorted feeds from multiple, unsynchronized, 
real-time video sources. 

The video controller 252 provides the video portion 262 of the hypermedia input 202 to a programmable 
low-pass filter 264, such as a Faraday FL676 or a Micro Linear ML6420, to remove high frequency aliasing 
components so as to form a filtered video signal 266. In turn, the filtered video signal 266 is fed to a programmable 
video scaler 268, such as a Phillips SAA 7140A or SAA 7186. The filtered video signal 266 improves the output 
quality of the video scaler 268, which enables resizing of a video image frame to a selected size for viewing and 
editing. In contrast to the RGB color signals of the prior art, the video scaler 268 outputs a scaled YUV video signal 
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comprising a luminance signal 270 and two color difference signals 272, 274. A gamma corrector 276 receives the 
signals 270, 272, 274 and performs gamma correction for the non-linear illumination of cathode ray tube (CRT) 
displays in addition to various color effects, such as posterization, negative imagery, tints and enhancements, in a 
preferred embodiment, the gamma corrector 276 is implemented as a lookup table in semiconductor memory. The 
5 resulting gamma corrected signals 278 are fed to a first-in, first-out (FIFO) buffer 280, then to an input frame 
controller 282, which assembles gamma corrected signals 278 into a video frame 284. A memory controller 286 
receives and transfers the video frame 284 to the media buffer 216 (Figure 5} for rapid access by the media editor 
210. In contrast to conventional memory controllers for PC based non linear editing systems, which access multiple 
devices using a wide variety of fetches, such as cache burst, DMA, and CPU instruction or data fetches, the memory 

10 controller 286 of the present invention is preferably optimized for multiple sequential fetches of large memory blocks. 

To perform a transition between a first and a second sequence of frames using a selected first frame and 
a selected second frame, each representing their respective frame sequences, the input frame controller 282 stores 
the first frame selected from the first frame sequence in a first location of the media buffer 216. Similarly, the 
input frame controller 282 stores the second frame selected from the second frame sequence in a second location 

15 of the media buffer 218. In response to the consumer's input, the non-linear editor 200 (Figure 5) creates an alpha 
frame, which describes how to combine the first and second frames so as to form the transition, and stores the 
alpha frame in a third location of the media buffer 216. An alpha frame is a video frame including an alpha value 
for each pixel, instead of a pixel value representing a color in a conventional video frame. The alpha value defines 
a mix level between corresponding pixels of the first and second frames. A SCRAM engine 288 retrieves the first 

20 frame, the second frame and the alpha frame from the media buffer 216. Upon retrieving these frames, the SCRAM 

engine 288 forms a transition frame, on a pixelated basis, by combining the pixels of the first, second and alpha 

frames according to the equation: 

Transition Frame pixel - [(First Frame pixel * Alpha Frame pixel) + Second Frame pixel * (1 • 
Alpha Frame pixel)], 

25 

where each pixel in an alpha frame is normalized to a value between zero and one inclusive. Lastly, the SCRAM 
engine 288 provides the transition frame to the memory controller 286. The memory controller 286 may transfer 
the transition frame to the media buffer 216 (Figure 5) or to the compression engine 212 (Figure 5), which 
compresses the transition frame data for subsequent transfer to the bus 214 (Figure 5) for capture in the storage 

30 222 (Figure 5). in a preferred embodiment, the media editor 210 includes the Applied Magic Video Processor. 

A consumer may set the pixel values for any of the first, second and alpha frames equal to a constant, 
indicating a single color for the entire frame. In response, the SCRAM engine 288 does not retrieve the frames, but 
instead uses the specified constant values to calculate the transition frame. For example, the transfer of an alpha 
frame typically requires a 10 Megabyte/sec bandwidth and the transfer of a first or second video frame typically 

35 requires a 20 Megabyte/sec bandwidth. To reduce bandwidth and media buffer memory, the SCRAM engine 288 
uses a single constant value for transitions, such as a dissolve or fade between a first and second frame sequence, 
requiring the same constant value for each pixel of the alpha frame. Thus, instead of using 10 Megabytes/sec of 
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bandwidth to transfer alpha frame data, the SCRAM engine 288 uses only 30 bytes/sec for NTSC video or 25 
bytes/sec for PAL video. Similarly, for a fade to black, the SCRAM engine 288 uses single constant values for both 
the alpha frame and the second frame to avoid the use of a further 20 Megabytes/sec of bandwidth to transfer pixel 
data for both the alpha frame and the second frame. Moreover, in a preferred embodiment, reductions in the 
bandwidth requirements of the present invention enable the use of a media buffer 216 (Figure 5) comprising 
inexpensive DRAM, instead of expensive dual port video RAM (VRAM). 

To play back video transitions, an alpha playback controller 290 communicates a fetch request to the 
memory controller 288 to retrieve selected alpha frames from the media buffer 216 (Figure 5). The alpha playback 
controller 290 provides the retrieved alpha frames to a Video Arithmetic Logic Unit (ALU) 292 to combine with the 
first and second frames so as to form transition frames. In a similar manner, a first playback controller 294 and 
a second playback controller 296 simultaneously retrieve the first and second frames from the media buffer 216 
through the memory controller 286. The retrieved frames are buffered in corresponding FIFOs 298, 300, 302 for 
ready access by the Video ALU 292. The Video ALU 292 uses the retrieved alpha frames to combine the first and 
second frames retrieved so as to form a transition frame for output to an encoder 206 (Figure 5) or to a viewing 
device. Moreover, for unmodified video, the present invention likewise reduces playback bandwidth and eliminates 
unnecessary memory fetches by disabling the second playback controller 296 and the alpha playback controller 290 
to free up 30 Megabytes/sec of bandwidth for other system uses. A similar 30 Megabyte/sec bandwidth reduction 
would result from fade to black during playback. The alpha playback controller 290 enables more complex playback 
features, such as real time rendering of flying or scrolling text overlays, moving pre-rendered graphical objets onto 
an image from a variety of hypermedia sources, and providing on-screen controls on top of the playing video. The 
Video ALU 292 receives input from three real-time video paths, the master pixel clock 250, an alpha path having 
mixing information for two full color video input paths, and serves as the primary video output of the media editor 
210. 

The media editor 210 also provides for the playback, capture and mixing of multiple audio channels, each 
channel having a left and right input with individual volume controls. An audio codec interface 304 receives the 
audio portion of the hypermedia input 202 (Figure 5) and outputs the audio portion of the hypermedia output 208 
(Figure 5). The audio codec interface 304 buffers audio samples for playback of all audio channels to an audio write 
FIFO 306. An audio write controller 308 transfers audio samples retrieved from the audio write FIFO 306 to the 
memory controller 286 for storage in contiguous memory locations in the media buffer 216 (Figure 5) or in the 
storage 222 (Figure 5) by transfer through the bus 214. To play back captured audio samples, an audio read 
controller 310 retrieves selected audio samples from the media buffer 216 (Figure 5) or the storage 222 (Figure 5) 
by issuing a fetch request to the memory controller 286. The audio read controller 310 buffers retrieved audio 
samples in an audio read FIFO 312, which communicates with an audio mixer 314, wherein volume information for 
each channel is added to audio samples retrieved from the audio read FIFO 312. Lastly, the audio mixer 314 
provides a mixed audio sample having channel volume information to the audio codec interface 304, which 
synchronizes the mixed audio sample with video for output using the master pixel clock 250. 
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The memory controller 286 manages the interface between the media editor 210 and the media buffer 216. 
In a preferred embodiment, the media buffer 216 comprises dynamic RAM (DRAM) memory, instead of costly dual 
port video RAM (VRAM). To satisfy DRAM timing and access constraints, the memory controller 286 provides 
arbitrated burst access to the media buffer 216 from the media editor 210. As multiple controllers within the media 
5 editor 210 access the media buffer 216, the memory controller 286 uses bank interleaving to improve access 
performance by avoiding page misses. Bank interleaving exploits the performance of the DRAM memory. By 
allocating video frames in a selected order, the memory controller 286 can perform fetches without page misses. 
One embodiment of a bank interleaving scheme which eliminates page misses and optimizes use of available memory 
bandwidth is as follows: 

10 1. Activate memory bank 0 with a row address from the alpha playback controller 290; 

2. Burst read the alpha frame data while activating Bank 1 with a row address 

from the second playback controller 296 for second frame data; 

15 3. Interrupt the transfer to the alpha playback controller 290 after 'x' cycles to 

transfer second frame data to the second playback controller 296; activate bank 0 again with 
a row address from the first playback controller 294; 

4. Burst read second frame data for '2x' cycles; 

20 

5. Burst read first frame data for '2x' cycles. 

Note that a bank interieaving scheme optimized to avoid page misses is dependent upon the number of concurrent 
active memory accesses, in a preferred embodiment, the memory controller 286 locates video frame data within 

25 the media buffer 216 so as to avoid page misses when retrieving video frame data. Moreover, the memory controHer 
286 prioritizes access to the media buffer 216. For example, when the alpha playback controHer 290, the first 
playback controller 294, the second playback controller 298 and the input frame controller 282 require real time 
access, the memory controller 296 sets the access priorities so as to ensure that each controller receives its data 
when needed. The memory controller 286 is preferably programmable so that the priority of access for each 

30 operational mode is selectable. Lastly, due to the relatively small bandwidth for audio access as compared to video 
access, the audio read controller 308 and the audio write controller 310 may access the media buffer 216 at all 
times. 

III. Operation of the Non-Linear Editing System 

A consumer may use the non-linear editing system 100 (Figure 1) to capture, manipulate, interact with, 

35 transport and play back hypermedia from a plurality of concurrent, independent hypermedia sources. Figure 7 
illustrates a method of operating the non-linear editing system 100. At state 360, a consumer initializes the system 
100 by powering on the main unit 122 (Figure 1). In a preferred embodiment, the consumer depresses the "On" 
button of the remote control 124 (Figure 1). During initialization, the system 100 may perform a power on self test 
(POST) to deternune if all of its components, such as the memory 220 (Figure 5), are functioning properly. 

40 Additionally, the processor 218 (Figure 5) loads an embedded operating system, preferably the Microware QS9000 
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for the IBM PowerPC 403, and software instructions for the GUI and non-linear editing functions from the memory 
220. In a preferred embodnment, the software instructions for the non-linear editing system 100 are stored in a 
programmable control ROM portion of the memory 220. In contrast to prior art systems, previously captured data 
is automatically loaded at state 382. Moreover, at state 362, the system 100 is preferably restored to its condition 
immediately prior to the last power down. Advantageously, a consumer may proceed with all tasks in progress as 
if the system 100 had not been powered down. At state 364, a determination is made whether to capture new 
data. If new data is not desired, the system 100 provides a storyboard having previously captured data at state 
372, wherein a consumer may edit and play back the storyboard. A storyboard is a sequential representation of 
a clip having video, audio, stills, effects and other hypermedia, presented as a picture icon. The sequence runs from 
left to right in a top to bottom order. If new data is desired, the system 100 checks the storage 222 (Figure 5) 
for available space at state 366. If there is no available space in the storage 222, the system proceeds to state 
368 to make space available. Otherwise, the system 100 captures hypermedia at state 370 and proceeds to state 
372 where the system 100 automatically provides a storyboard having the captured hypermedia. Lastly, at state 
374, a consumer may create a copy of a storyboard to a recordable DVD or videotape. 

Referring now to Figure 8, a flowchart illustrates a method for manipulating hypermedia. At state 380, 
a consumer captures hypermedia to a storyboard. The storyboard often includes a plurality of captured hypermedia 
portions, or clips. At state 382, the consumer manipulates the captured hypermedia on the storyboard. The non- 
linear editing system 100 (Figure 1) preferably includes selected default transitions between each pair of clips 
automatically. For example, consumers manipulate clips by enhancing them with graphics, text and audio annotations. 
Consumers may likewise enhance electronic mail (e mail) by excerpting, editing and attaching the edited clips to their 
e-mail messages. Consumers also manipulate clips by compiling a sequence of clips and creating the transitions 
between each pair of clips. Additionally, consumers may manipulate clips by incorporating digitized photographs, 
synchronized music and other forms of digital hypermedia captured from the Internet via the Internet browser 
functionality incorporated into the non linear editing system 100. For example, a consumer may manipulate a clip 
by excerpting a video still or a digital photograph from a cfip for placement on a World Wide Web page or print out 
as a photograph or postcard. At state 384, the consumer may modify the default transitions selected by the non- 
linear editing system 100. Moreover, at state 386, the system 100 enables consumers to add overlays, such as 
graphical and audio annotations of clips. Lastly, consumers may play back their storyboards at state 388 or copy 
their storyboards at state 390. Consumers preferably copy the hypermedia represented by storyboards to videotape 
machines, such as a VCR or camcorder. 

Figure 9 illustrates an embodiment of a graphical user interface 400 useful for the capture of hypermedia. 
The capture GUI 400 provides a plurality of tabs 402 enabling selection of various functions of the non-linear editing 
system 100. In capture mode, the capture GUI 400 preferably includes a shot tab 404 displaying icons 406 
representing previously captured clips. The capture GUI 400 likewise includes a display window 408 to display an 
image from a clip referenced by a selected icon 410. Moreover, the capture GUI 400 includes a control panel 412 
displaying a plurality of buttons 414 representing capture functions, such as record, play, stop and pause. Similarly, 
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the capture GUI 400 provides for the selection of capture properties using at least one drop down menu 416. For 
example, using the drop down menu 416, a consumer may instruct the non-linear editing system 100 to capture from 
a VCR to Storyboard 1 as illustrated in Figure 9. The capture GUI 400 may also include a selection box 418 to 
specify hypermedia data types to capture as well as an entry box 420 to specify a clip name. The capture GUI 400 
may additionally include a quality window 422 having a slide bar 424 and a slider 426. The consumer selects the 
capture quality by manipulating the slider 426 within the slide bar 424. Lastly, the capture GUI 400 includes an 
icon bar 428 having icons representing the different modes of the non linear editing system 100. The consumer 
manipulates the capture GUI 400 using the point and click functionality of the pointing device 134 (Figure 2b) to 
select and manipulate items within the capture GUI 400 in coordination with the keyboard 130 (Figure 2b) to provide 
keyboard inputs as needed. 

Figure 10 illustrates an embodiment of a storyboard GUI 440. As discussed previously, the storyboard GUI 
440 includes a shot tab 404 displaying icons 406 representing previously captured clips. The storyboard GUI 440 
likewise includes a display window 408 to display an image from a clip referenced by a selected icon 410. In 
addition, the storyboard GUI 440 includes a plurality of storyboard tabs 442, each tab referencing a selected 
storyboard having a sequence of clips with transitions. For example, a vacation tab 444 includes a vacation 
storyboard 446 having six clips, corresponding to the image icons, and five transitions, one transition between each 
pair of clips. The vacation storyboard 446 includes a time ruler 448 having a graphics bar 450, a first audio bar 
452 and a second audio bar 454. The bars 450, 452, 454 indicate the name and duration of the hypermedia portion 
corresponding to the clip represented by the icon above the bar. Each of the storyboard tabs 442 includes a scroll 
bar 456 to scroll down the window illustrating the storyboard corresponding to the selected tab. To add a clip to 
a storyboard, the consumer selects the shots tab 404 and one of the storyboard tabs 442 of interest, such as the 
vacation tab 444, to display the storyboard 446. The consumer then selects, drags and drops a clip icon 410 onto 
the storyboard 446 as shown in Figure 10. The non-linear editing system 100 automatically provides a default 
transition on the storyboard immediately prior to the newly added clip 410. Lastly, the storyboard GUI 440 includes 
a button panel 458 having a plurality of buttons corresponding to a wide variety of editing functions for the 
storyboard 446, such as save, print, cut, copy and paste. A consumer may edit a storyboard 446 using the point 
and click functionality of the storyboard GUI 440 in conjunction with the functions corresponding to buttons 
comprising the button panel 458. 

The storyboard GUI 440 preferably displays the sequence of items on a storyboard instead of their absolute 
time scales. Thus, icons of a single size represent each captured clip, regardless of its actual duration. However, 
other data on the storyboard associated with a captured clip is represented by an indicator having a size and location 
corresponding to the temporal relationship with the captured clip. For example, an audio clip having a duration of 
half the length of a captured clip appears on a first 452 or second 454 audio bar under the icon of the captured 
clip with an indicator bar half the length of the icon. Moreover, for a half length audio clip synchronized to begin 
at one quarter of the clip duration, the indicator bar appears indented one quarter of the icon width from both sides, 
corresponding to a start at one quarter clip duration and a termination at three quarter clip duration. Thus, the time 
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ruler 448 of the present invention, referred to as a cartoon timeline, illustrates the sequential relationship of events. 
Advantageously, the cartoon timeline metaphor eliminates the need for consumers to zoom in on various portions of 
the storyboard to identify and manipulate clips, thereby improving the usability of a story board GUI 440. Similarly, 
the cartoon timeline metaphor of the present invention eliminates the need for a high resolution computer graphics 
monitor as a high quality image of the storyboard GUI 440 can be rendered on a conventional television set. 
Moreover, the cartoon timeline metaphor eliminates the need to render a minute sliver of an icon, representing a 
captured clip of extremely short duration, on a low resolution television screen for consumer viewing from relatively 
large distances. Lastly, in contrast to prior art systems, the cartoon timeline metaphor of the present invention 
illustrates the relationship between associated data, such as audio and graphical annotations, and the captured clips 
through indicator bars, which are relative in nature as opposed to absolute in time. 

Figure 1 1 illustrates an embodiment of a transition GUI 470. As shown in the storyboard GUI 440 (Figure 
10), the transition GUI 470 includes a vacation tab 444 having a vacation storyboard 446 and a button panel 458 
having a plurality of buttons corresponding to editing functions. In the transition GUI 470, the transitions tab 472 
is selected to display a plurality of icons 474, each icon representing a predetermined transition effect. As before, 
to change a transition, the consumer selects, drags and drops a desired transition icon 476 on the storyboard 446 
at a location between a pair of clips. The consumer may likewise edit the transitions of a storyboard 446 using 
the point and click functionality of the transition GUI 470 in conjunction with the functions corresponding to button 
comprising the button panel 458. Note that the transitions tab 472 includes a scroll bar to scroll down the window 
illustrating the icons corresponding to various transition effects. 

Figure 12 illustrates an embodiment of a graphics GUI 490. The graphics GUI 490 includes a graphics tab 
492 having a scroll bar 494 and a plurality of icons 496 representing various graphics overlays, such as color, titles, 
and text on color. The graphics GUI 490 similarly includes a display window 408 to display an image for graphical 
editing. The graphics GUI 490 also includes an image edit window 498 having a plurality of image editing tool 
buttons 500, such as a line tool, a box tool, a text tool, cut, copy and paste, about the periphery of the image edit 
window 498. As before, a consumer uses the point and click functionality of the remote control 124 (Figure 2b) 
to select and manipulate the image editing tool buttons 500. Moreover, the graphics GUI 490 includes a features 
portion 502. The features portion 502 includes an name box 504, wherein the consumer may select a name to 
identify the edited graphic and a plurality of tabs 506, each tab having property selections for some of the image 
editing tool buttons 500. For example, a color tab 508 includes selection boxes 510 to select color features of lines, 
fills and shadows. Lastly, the features portion 502 includes a pair of slider bars 512, each having a slider 514, for 
the selection of color gradients. Operation of the graphics editor is similar to conventional graphics editors, such 
as MacPaint and Paintbrush. 

Each major element of the GUI is derived from a list of consumer tasks. Figure 13 is a task flow diagram 
550 illustrating the manipulations a consumer may perform with the non linear editing system 100. Each of the 
illustrated tasks is intra-cyclical in nature, meaning that a consumer may perform the task over and over until 
satisfied with the results. Note that illustrated tasks are also inter-cyclical in that a consumer may pass forward 
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and back to other tasks. As discussed previously, the software functionality for carrying out the tasks is located 
in the processor 218 (Figure 5), which loads the software instructions from the memory 220 (Figure 5). At 
initialization, the system automatically loads previously captured hypermedia portions from the storage 222 (Figure 
5) and places them into a storyboard so that a consumer may create an edited storyboard. At task 552, a consumer 
may capture, or digitize for storage, hypermedia portions, or shots, from a hypermedia source. Ultimately, the 
hypermedia shots are stored in the storage 222 (Figure 5). During this task, a consumer may perform some "rough" 
editing by electing not to digitize selected portions that they do not wish to include in the finished master. At task 
554, the consumer places captured shots on the storyboard. The storyboard may be thought of as a working portion 
of the media buffer 216 (Figure 5). As discussed previously, shots are automatically placed into the storyboard in 
the order in which they were captured. Similarly, when shots are placed into the storyboard, transitions between 
shots are automatically generated. At task 556, the consumer may perform "fine" editing of shots, such as moving, 
duplicating, removing and manipulating the duration of shots. For example, in a storyboard having 5 shots with 4 
interposed transitions, a consumer may move the first shot to the end, whereby the system automatically removes 
the transition between the first and second shots and adds a default transition between the end shot and the first 
shot. The consumer may likewise modify the transitions at task 558 and annotate shots with overlays, such as 
graphics and audio, at task 560. Moreover, the consumer may playback the storyboard at task 562 or may save 
a copy of the storyboard at state 564. When the consumer has completed all desired tasks on this storyboard, the 
consumer may create another storyboard at task 566. 

Figure 14 is a diagram illustrating the GUI display flow. The GUI is implemented by a non-linear or event 
based software application. In traditional user interface development, the diagram would represent the menu 
structure of the application. However, in a presently preferred embodiment, the first layer 580 identifies the major 
system functions represented by icons 428 (Figure 9). The second layer 582 identify further functions and features 
corresponding to the major system functions of the first layer 580. The capture function 584 provides tools to 
capture from a variety of hypermedia sources, such as a camcorder 586, a VCR 588 and a digital camera 590. The 
storyboard function 592 provides tools to edit, playback and record shots. Storyboard tools include graphics 594, 
transitions 596 and effects 598. The paint box function 600 provides additional editing tools for drawing 602, 
character generation 604, video underlay 606 and still photo underlay 608. The Internet function 610 includes tools 
for e mail 612 and a browser plug-in 614, wherein commercially available browsers, such as Internet Explorer and 
Netscape Navigator, may be integrated. A consumer may add, remove, change or otherwise manipulate hyperlinks 
using the browser. The set-up function 618 enables configuration of the system by providing tools to indicate 
preferences 620 and software installation/removal tools 622. The extras function 624 provides extendibility tools 
to supplement and extend system functions. Lastly, the heH> function 628 provides tools to access system help 
documentation. 

All objects, such as shots, audio, graphics and transitions, can be manipulated by initiating a details display 
from the button panel 458 (Figure 10) after selecting the desired object from the storyboard. The consumer can 
play the edited hypermedia in a playback area or full screen at any time during the editing process, and when 
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satisfied, the consumer may record the edited hypermedia to an output device, such as a recordable DVD or VCR, 
by selecting the Select All button on the storyboard display and then selecting the Master button. Thus, rather than 
a delivery only system for hypermedia, the present invention allows the consumer to, for example, download 
hypermedia from the Internet and edit or manipulate it through the storyboard program. 
IV. Summary 

The present invention provides a complete, standalone, economical system that enables consumers to create, 
manipulate, edit, view and output hypermedia in their homes. In contrast to the prior art, the non-linear editing 
system of the present invention does not require a consumer to incur substantial expenses to purchase a personal 
computer having expansion bus add-in cards and software to capture and edit hypermedia in their homes. Similarly, 
unlike current personal computers, the present invention is adapted for communication with conventional home 
entertainment components, such as a VCR, television, stereo, camcorder and digital still camera. Moreover, the non- 
linear editing system of the present invention provides for a wide variety of hypermedia output mediums, such as 
computer displays, television, printers, videotapes, DVDs, and audio speakers. 

In addition, the present invention advantageously overcomes the expense and complexity of prior art non- 
linear editing systems by providing an inexpensive and easy to learn system for home use. Unlike conventional non- 
linear editing systems designed for professional use, the present invention automatically loads previously captured 
data onto the storyboard for editing. Similarly, the present invention automatically provides default transitions 
between each pair of clips on the storyboard. The present invention thus simplifies and facilitates the editing of 
home movies by enabling consumers to immediately manipulate captured clips, instead of undertaking complicated 
tasks to configure the storyboard as required by conventional non-linear editing systems. The present invention 
likewise provides novel methods of data handling to reduce bandwidth requirements and leverages the availability of 
low cost components to perform the functions of the costly, specialized components of the prior art. The media 
editor of the present invention combines the functions of various prior art software and hardware components into 
a novel architecture, which is integrated onto a low cost application specific integrated circuit (ASIC), The novel 
architecture of the present invention permits the use of inexpensive DRAM to create hypermedia effects using 
multiple video and analog channels simultaneously. In contrast to costly prior art VRAM buffers, the media editor 
of the present invention provides for priority based interleaved access to a DRAM based media buffer that delivers 
full video bandwidth. Additionally, the system accepts a wide variety of hypermedia data types and stores them 
in the same storage, enabling the mixing of audio, video, text, music and other data types using common mixing 
engines. Moreover, the system allows for the capture of multiple hypermedia channels with simultaneous playback 
and editing. Furthermore, because the hypermedia output is not dependent on the hypermedia input synchronization, 
hypermedia inputs may be removed, attached and switched on the fly, resulting in different hypermedia input 
synchronizations without any effect on the timing or quality of the hypermedia output. Lastly, audio is synchronized 
to video by reference to a master pixel clock, which is also used to prevent "tearing" and other distortions. 

The invention may be embodied in other specific forms without departing from its spirit or essential 
characteristics. Thus, the described embodiments are, in all respects, merely illustrative and not restrk:thre. 
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Moreover, the appended claims, rather than the foregoing description, serve to define the scope of the invention. 
These claims embrace within their scope all changes which come within their meaning and range of equivalency. 



wo 98/06098 PCT/US97/13055 

-17- 

WHAT IS CLAIMED IS : 

1. A non-linear editing system, comprising: 
a remote control to provide command signals; 

a main unit, responsive to the command signals, having a non-linear editor to receive, access, and edit 
hypermedia and a direct access storage device to capture the hypermedia; and 

an output device in communication with the main unit to receive and play the hypermedia. 

2. The non-linear editing system of Claim 1, wherein the remote control comprises: 
a power system; 

a keyboard having at least one button; and 

a keyboard processor in communication with the keyboard and receiving power from the power system, 
wherein the keyboard processor receives and converts input from the keyboard into the command signals. 

3. The non-linear editing system of Claim 2, further comprising at least one pointing device in 
communication with the keyboard processor, wherein the keyboard processor receives and converts input from the 
at least one pointing device into the command signals. 

4. The non-linear editing system of Claim 3, wherein the at least one pointing device may be 
configured to operate as a jog/shuttle wheel and wherein the keyboard processor receives and converts input from 
the jog/shuttle wheel into the command signals. 

5. The non-linear editing system of Claim 2, further comprisaig a jog/shuttle wheel in communication 
with the keyboard processor, wherein the keyboard processor receives and converts input from the jog/shuttle wheel 
into the command signals. 

6. The nonlinear editing system of Claim 2, further comprising a wireless transmitter in 
communication with the keyboard processor, wherein the keyboard processor provides the command signals to the 
wireless transmitter for conversion into a wireless command signal. 

7. The non-linear editing system of Claim 1, wherein the output device comprises a television set. 

8. The non-linear editing system of Claim 1, wherein the output device comprises a computer display. 

9. The non-Nnear editing system of Claim 1, wherein the output device comprises a videotape 

recorder. 

10. The non-linear editing system of Claim 1, wherein the output device comprises at least one 

speaker. 

11. The non-linear editing system of Claim 1, wherein the output device comprises a recordable CD- 

ROM. 

12. The non-linear editing system of Cbim 1, wherein the output device comprises a DVD. 

13. The non linear editing system of Claim 1, wherein the output device comprises a printer. 

14. The non-linear editing system of Claim 1, wherein the main unit communicates with a network 
to access the hypermedia. 
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IS. The non-linear editing system of Claim 14, wherein the network provides access to a real-time 
hypermedia source. 

16. The non-linear editing system of Claim 14, wherein the network provides access to an off-line 
hypermedia source. 

5 17. The non-linear editing system of Clakn 14, wherein the network comprises a broadcast television 

network. 

18. The non linear editing system of Claim 14, wherein the network comprises a cable television 

network. 

19. The non linear editing system of Claim 14, wherein the network comprises a satellite television 

10 network. 

20. The non-linear editing system of Claim 14, wherein the network comprises the World Wide Web 
portion of the Internet. 

21. The non-linear editing system of Claim 1, wherein the main unit accesses the hypermedia from 
a videotape player. 

15 22. The non-linear editing system of Claim 1, wherein the main unit accesses the hypermedia from 

a radio tuner. 

23. The non-Hnear editing system of Claim 1, wherein the main unit accesses the hypermedia from 
a compact disc player. 

24. The non-linear editing system of Claim 1, wherein the main unit accesses the hypermedia from 
20 a CD-ROM. 

25. The non linear editing system of Claim 1, wherein the main unit includes a communication port. 

26. The non-linear editing system of Claim 25, wherein the communication port comprises an RS-232 
serial port. 

27. The non-Knear editing system of Claim 25, wherein the communication port comprises a parallel 

25 port. 

28. The non-linear editing system of Claim 25, wherein the communication port emprises an IrDA 
compHant infrared port. 

29. The non-linear editing system of Clamt 25, wherein the communication port comprises a PCMCIA 

port. 

30 30. The non-linear editing system of Claim 25, wherein the communication port comprises a USB port. 

31. The non linear editing system of Claim 25, wherein the communication port comprises a SCSI port. 

32. The non-linear editing system of Clam 1, wherein the hypermedia is in a compressed digital data 

format. 

33. A non-linear editor, comprising: 
35 a bus; 

a processor, in communication with the bus, to control access to the bus; 
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a memory in communication with the processor; 

a compression engine, in communication with the bus, to compress and decompress hypermedia; 
a storage, in communication with the bus, to capture and provide direct access to the compressed 
hypermedia; and 

a media editor, in communication with the compression engine and the bus, the media editor and the 
compression engine receiving the hypermedia, wherein the media editor manipulates the hypermedia. 

34. The non-linear editor of Claim 33, further comprising a decoder in communication with the 
compression engine and the media editor, wherein the decoder receives and decodes a hypermedia input so as to form 
the hypermedia. 

35. The non linear editor of Claim 34, wherein the decoder includes an analog to digital converter. 

36. The non linear editor of Claim 33, further comprising an encoder in communication with the media 
editor, wherein the encoder recehfes and encodes the hypermedia from the media editor so as to form a hypermedia 
output. 

37. The non-linear editor of Claim 36, wherein the encoder includes a digital to analog converter. 

38. The non-linear editor of Claim 33, further comprising a media buffer in communication with the 
media editor. 

39. The non-linear editor of Claim 38, wherein the media buffer comprises dynamic random access 

memory. 

40. The non-linear editor of Claim 33, wherein the processor comprises an embedded controller. 

41. The non-linear editor of Claim 33, wherein the memory includes a read only memory device. 

42. The non-linear editor of Claim 33, wherein the compression engine provides MPEG-1 compression 
and decompression. 

43. The non-linear editor of Claim 33, wherein the compression engine provides MPE6-2 compression 
and decompression. 

44. The non-linear editor of Claim 33, wherein the compression engine provides wavelet compression 
and decompression. 

45. The non-linear editor of Claim 33, wherein the compression engine provides JPEG compression and 
decompression. 

46. The non-linear editor of Claim 33, wherein the storage comprises a magnetic disk drive. 

47. The non-linear editor of Claim 33, wherein the storage comprises a DVD device. 

48. The non-linear editor of Claim 33, wherein the storage comprises a magneto-optic disk drhre. 

49. The non-linear editor of Claim 33, wherein the storage comprises a rewriteable optical disk drive. 

50. A method of operating a non-linear editing system, comprising the steps of: 
capturing hypermedia; 

automatically loading the previously captured hypermedia; and 
providing a storyboard to manipulate the previously captured hypermedia. 
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51. 



The method of Claim 50, further comprising the step of initializing the non linear editing system. 
The method of Claim 50, further comprising the step of making a copy of the storyboard. 



52. 



53. The method of Claim 50, further comprising the steps of: 
determining if a storage has space to capture a new hypermedia portion; 

if the storage does not have space, making the space available in the storage; and 
capturing the new hypermedia portion to the storyboard. 

54. The method of Claim 50, wherein the step of automatically loading the previously captured 
hypermedia includes providing a default transition between a pair of previously captured hypermedia portions. 

55. A method of editing hypermedia in a non-linear editing system, comprising the steps of: 
capturing a plurality of hypermedia portions to a storyboard; and 

automatically providing a transition between a parr of hypermedia portions within the plurality of hypermedia 
portions on the storyboard. 

56. The method of Claim 55, further comprising the step of manipulating at least one of the plurality 
of hypermedia portions. 

57. The method of Claim 55, further comprising the step of modifying the transition. 

58. The method of Claim 55, further comprising the step of adding an overlay to at least one of the 
plurality of hypermedia portions. 

59. The method of Claim 55, further comprising the step of playing back the storyboard. 

60. The method of Claim 55, further comprising the step of copying the storyboard. 
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